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ABSTRACT:  The  Probability  of  Raid  Annihilation,  PRA,  is  the  measure  of  a  single  ship  with  its  combat  systems  to 
detect,  control,  engage  and  defeat  a  specified  raid  of  threats  within  a  specified  level  of  probability  in  an  operational 
environment.  The  Navy,  led  by  Program  Executive  Office  for  Theater  Surface  Combatants  (PEO  TSC),  is  developing  an 
assessment  process  for  PRA  that  uses  an  innovative  combination  of  live  testing  and  a  simulation,  the  PRA  Federation.  A 
crucial  element  in  the  assessment  process  is  identifying  the  uncertainties  in  the  PRA  measure,  in  particular  those 
associated  with  the  natural  environment. 

The  PRA  Federation  is  unusual  in  that  a  given  threat  reacts  to  the  combined  ship  defenses  while  all  ship  defenses 
simultaneously  see  the  common  threat.  This  complexity  of  interactions  with  the  associated  environmental  effects  makes 
it  very  difficult  to  determine  uncertainties  in  the  due  to  environmental  considerations.  To  further  complicate 
matters,  the  Federation  is  designed  to  permit  the  interchange  between  operational  and  test  scenarios,  thereby 
supplementing  live  testing  with  simulation  results.  This  interchange  capability  demands  that  the  relation  between  the 
real  world  and  implemented  environment  be  captured  in  detail  so  that  resulting  uncertainties  in  the  due  to  the 
environment  can  be  well-understood  and  documented. 

Using  the  PRA  measure  as  a  specific  example,  this  paper  illustrates  a  new  technique  in  evaluating  how  the  outcome  of 
simulations  is  affected  by  uncertainties  in  the  natural  environment  provided.  The  Environment  Concept  Model  (ECM) 
process,  developed  under  the  Navy  MARitime  Environmental  Data  Standards  (MARVEDS)  program  for  use  in 
developing  environmental  requirements,  is  applied  to  identify  and  document  the  complexities  within  the  Federation 
and  the  links  with  the  operational  and  test  environments.  In  particular,  the  assumptions  and  constraints  in  modeling  the 
environmental  effects  are  captured  in  detail  to  assess  the  appropriate  level  of  fidelity  and  consistency  required  in  the 
environmental  representation.  In  other  words,  the  ECM  process  links  the  simulation  outcome,  the  PRA  measure,  back  to 
the  required  natural  environment  representation.  Using  the  same  procedure,  the  uncertainty  in  the  measure  can  be 
linked  mathematically  back  to  the  uncertainty  in  the  required  natural  environment  representation  and  vice  versa.  Thus, 
the  ECM  process  and  documentation  capture  the  traceability  needed  not  only  to  determine  natural  environment 
representation  requirements  but  also  to  evaluate  how  uncertainty  in  that  environmental  representation  affects  the 
uncertainty  simulation  outcome. 


1.  Introduction 

This  paper  describes  a  promising  methodology  to  assess 
uncertainties  in  the  Probability  of  Raid  Annihilation  (Pra) 
Measure  of  Effectiveness  (MOE)  used  by  the  Navy  as 
part  of  the  assessment  for  ship  self  defense.  The 


methodology  builds  on  a  number  of  tools,  techniques  and 
programs  in  Modeling  &  Simulation  (M&S).  These, 
including  MARVEDS,  are  described  very  briefly  in 
Section  2,  Background,  together  with  references  for 
additional  information.  The  PRA  process  and  the  role  that 
the  ECM  plays  are  described  in  Section  3.  The  new  aspect 
of  ECM,  in  addition  to  capturing  environmental 
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requirements  and  as  a  basis  for  validation,  is  provided  in 
Section  4  with  Conclusions  following  in  Section  5. 

2.  Background 

The  Navy  Modeling  &  Simulation  Office  (N6M)  has 
established  the  MARitime  Environmental  Data  Standards 
(MARVEDS)  program  to  develop  and  promote  standards 
for  representing  and  using  the  integrated  natural 
environment  in  simulations.  Standards  can  include  aspects 
of  data  representations  such  as  parameters,  spatial  and 
temporal  resolutions,  formats  and  coordinate  systems. 
Standards  can  also  address  serving  methods  for  delivering 
dynamic  data  and  even  best  practices  for  applying  such 
data  in  effects  models.  The  development  of  such 
standards  is  based,  in  part,  on  the  requirements  for  the 
integrated  natural  environment  in  Naval  M&S.  However, 


existing  requirements  do  not  provide  the  level  of  detail 
necessary  to  develop  such  standards.  Therefore, 
MARVEDS  is  developing  a  formalism,  including  a  living 
document,  to  capture,  collate,  and  coalesce  the  integrated 
natural  environment  requirements  for  the  Navy  programs 
and  offices  involved  in  developing  simulations.  This 
formalism  includes  the  Environment  Concept  Model 
(ECM),  a  procedure  and  a  process,  to  unambiguously 
describe  the  integrated  natural  environment  information 
needed  by  a  simulation  and  the  how  that  information  is 
used  in  the  simulation  itself.  The  role  that  MARVEDS 
plays  in  M&S  is  graphically  presented  in  Figure  1,  which 
is  a  variation  of  the  Environmental  Reference  Model.  For 
further  information,  contact  Dr.  S.  K.  Numrich,  the 
MARVEDS  Program  Manager,  at 
Numrich@ait.nrl.navy.mil. 


MARVEDS  mediates... 

-Between  providers  and  users 
-Across  data  and  models 
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Figure  1.  The  Role  of  MARVEDS  in  Simulation  Development 


The  formalism  being  developed  by  MARVEDS  is  based 
on  conceptual  modeling,  a  common  technique  for 
representing  information  systems,  especially  ones 
involving  complex  or  ill-defined  problems.  In  conceptual 
modeling  the  system  is  viewed  as  objects  that  can  interact 
in  various  ways.  Each  object  has  associated  properties, 
not  immutable,  that  determine  its  interactions  with  other 
objects.  The  process  of  developing  a  conceptual  model 
then  involves  determining  how  a  system  can  be  expressed 
in  terms  of  objects,  properties  and  interactions.  The 
concept  model  provides  a  common  basis  by  which 


stakeholders  and  developers  can  agree  on  the  purpose  and 
goals  of  the  system.  In  particular,  conflicts  and 
inconsistencies  can  be  identified  and  resolved.  The 
concept  model  should  capture  the  agreements  and 
decisions  that  then  lead  to  the  requirements  for  the 
system.  The  requirements,  in  turn,  form  the  basis  of  the 
validation  process  for  the  system.  (See  Reference  1  for 
more  information  on  conceptual  modeling.) 

Conceptual  modeling  has  many  applications  and  plays  an 
integral  role  in  the  FEDEP  process.  DMSO  has  invested 


in  the  Concept  Model  of  the  Mission  Space  (CMMS), 
now  the  Functional  Description  of  the  Mission  Space 
(FDMS).  The  documentation  for  the  FEDEP  and  the 
CMMS  do  not  address  the  level  of  detail  needed  to 
capture  the  integrated  natural  environment  requirements 
as  needed  to  handle  sensor  effects  in  Navy  M&S.  These 
and  other  DMSO  programs  are  found  on  the  DMSO  Web 
Site:  http://www.dmso.mil 

The  MARVEDS  Program  and  the  Integrated  Ship 
Defense  (ISD)  M&S  Pilot  Program  collaborated  to 
achieve  a  consistent  natural  environment  representation 
across  federates  that  use  legacy  codes.  (See  Reference  4.) 
The  representation  is  said  to  be  internally  consistent  if  it 
has  been  developed  in  accordance  with  known  physical 
and  dynamical  constraints.  Internal  consistency  is 
commonly  imposed  on  the  pre-runtime  integrated  natural 
environment  representation  provided  as  input  for  a 
simulation  exercise.  Achieving  and  maintaining  a 
consistent  integrated  natural  environment  during  runtime 
for  a  simulation  or  federation  is  much  more  complex. 
Legacy  codes  frequently  have  embedded  environmental 
data.  One  code  may  use  a  constant  wind  of  five  m/s  from 
the  north  while  another  uses  a  wind  of  one  m/s  blowing 
from  the  east.  Further,  the  pre-runtime  environment  data 
may  supply  a  wind  that  varies  temporally  in  magnitude 
and  direction.  Thus,  the  winds  in  the  legacy  codes  and  in 
the  pre-runtime  data  are  inconsistent  with  each  other. 

Conflicts  in  the  legacy  codes  have  to  be  resolved  to 
achieve  consistency  in  the  environment  but  first  such 
conflicts  have  to  be  identified  and  documented.  (See 
Reference  4  for  examples.)  Concept  modeling  is  ideal  for 
this  purpose  and,  further,  is  already  in  use  by  the  M&S 
community. 

The  ECM  is  a  process-based  product  in  the  form  of  an 
electronic  document.  The  resulting  document  is  composed 
of  three  major  sections.  First,  the  document  describes  the 
real  world  scenario(s)  the  simulation  will  represent.  The 
military  systems  and  personnel  are  identified,  as  well  as 
the  actions  they  perform.  Often,  the  scenarios  are  broken 
down  into  smaller  vignettes,  to  clarify  the  representations. 
The  document’s  second  section  describes  the  land/sea/air 
modeling  needed  to  satisfy  all  the  needed  terrain,  weather 
and  ocean  phenomena.  The  ECM  documents  the 
independent  and  dependant  variables,  algorithms  and 
logic.  The  document’s  third  section  revisits  the  second 
section’s  need-based  description,  from  a  different 
perspective.  The  third  section  describes  the  land/sea/air 
modeling  as  actually  implemented,  with  limitations 
imposed  by  science  understanding  as  well  as  project 
schedule  and  budget  considerations.  (See  Reference  3  for 
more  information  on  the  ECM.) 


We  use  the  Unified  Modeling  Language  as  the  analysis 
and  design  language  to  describe  environment 
representations  in  the  ECM.  Reference  5  provides  an 
introduction  to  this  standards-based  analysis  and  design 
language,  and  Reference  2  provides  a  detailed  user  guide. 

The  extensive  ECM  documentation  also  includes  details 
of  simulation  objects,  models,  assumptions,  constraints, 
agreements  and  other  related  reference  material.  As  such, 
this  documentation  can  provide  the  basis  for  validation  of 
the  system.  (See  Reference  6.) 

3.  Probability  of  Raid  Annihilation  (Pra) 

3.1  Definition 

The  Probability  of  Raid  Annihilation  is  defined  as  the 
ability  of  a  particular  stand-alone  ship,  as  an  integrated 
system,  to  detect,  control,  engage  and  defeat  a  specified 
raid  of  anti-ship  missile  (ASM)  threats  with  a  specified 
level  of  probability  in  the  operational  environment.  The 
Pra  MOE  is  a  system  of  systems  measure  that  is  levied  on 
the  ship  defense  suite  as  a  whole  to  properly  detect, 
control,  and  engage  (annihilate)  a  raid  of  incoming  threat 
ASMs.  The  ability  to  directly  test  the  capability  of  a  ship 
to  withstand  a  raid  is  not  practical.  Therefore,  a 
combination  of  live  tests  and  simulations  will  be  used.  In 
fact,  the  Pra  Federation  is  unique  in  that  simulation  will 
be  able  to  incorporate  directly  the  results  from  the  live 
testing. 

In  addition  to  the  PRA  metric,  an  evaluation  is  needed  of 
the  causes  of  uncertainties  in  the  system,  including  an 
estimate  of  the  uncertainty  in  the  PRA  metric  itself.  Models 
and  simulations  are  by  definition  subsets  of  reality 
because  of  the  constraints,  and  assumptions  imposed. 
These  assumptions  and  constraints  introduce 
uncertainties.  However,  live  testing  is  also  subject  to 
uncertainties  introduced  by  limited  scenarios  and 
observational  errors.  This  paper  describes  a  process  by 
which  these  uncertainties  can  be  documented,  analyzed 
and  evaluated  to  ascertain  their  impact  on  the  final 
outcome  of  the  determination  of  the  Probability  of  Raid 
Annihilation. 

The  Navy,  led  by  PEO  TSC,  has  convened  a  task  group  to 
evaluate  the  PRA  process  and,  specifically,  the 
environmental  effects  that  must  be  considered.  This  task 
group  includes  subject  matter  experts  in  the  areas  of  ship 
defense,  threats,  radars,  electronic  warfare  and  the  natural 
environment.  This  task  group  has  collaborated  to  define 
the  model  of  the  PRA  displayed  here.  Starting  with  a 
simple  framework,  as  shown  in  Figure  2,  additions  have 
been  made  so  that  an  increasingly  detailed  system  has 
been  captured  in  the  context  of  this  simple  model. 


Figure  2 


3.2  Current  application 

Figure  3  shows  more  detail  in  the  PRA  simulation.  Again, 
however,  the  detail  is  shown  only  for  one  subsystem,  the 
radar.  The  overall  system  consists  of  the  RAM  missile 
and  associated  launch  and  guidance  systems,  decoy 
systems,  and  electronic  warfare  systems.  A  complete 
evaluation  of  system  performance,  system  uncertainties, 
and  the  effects  of  the  environment  require  detailed 
information  on  all  of  these  subsystems. 

The  primary  goal  of  this  work  is  to  document  the 
assumptions  in  the  development  of  the  simulation  from  a 
description  of  the  real  world.  The  first  step  of  this  is  to 
compile  a  list  of  the  various  effects  that  can  act  on  the 
system.  Next,  it  is  necessary  to  either  incorporate  the 
effect  or  explain  and  document  why  it  couldn’t  or 
shouldn’t  be  included.  Table  1  contains  such  a  list  of 
environmental  effects  that  could  impact  a  radar  or  RF 
seeker.  (This  information  is  displayed  in  table  form,  rather 
than  in  the  Rational  Rose  Representation  used  in  other 
Figures,  to  save  space.) 


Only  the  sea  clutter  and  propagation  effects  are  retained 
for  evaluation,  primarily  because  appropriate  computer 
models  of  the  other  effects  are  not  readily  available. 
However,  the  other  effects  are  included  in  the  overall 
concept  model  with  explanations  regarding  why  they  have 
not  been  included,  facilitating  later  upgrades  and  reuse. 
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Figure  3 


The  environmentally  relevant  parameters  for  the  radar  are 
shown  in  Figure  4.  The  arrows  with  the  small  heads 
define  sources  of  information  for  an  object.  For  example 
the  “Radar”  object  will  request  range  information  from 
the  “RFPropagation”  object  and  sea  clutter  from  the 
“SeaClutter”  object.  In  turn  the  “SeaClutter”  object  will 
request  sea  state  information  from  the  “SeaState”  object. 


For  the  purposes  of  this  paper  the  uncertainties  will  be 
traced  from  the  determination  of  temperature  profile  and 
hence  radar  propagation  performance  to  the  impact  on  the 
final  determination  of  Pra  for  this  ship.  It  is  understood 
that  this  process  must  be  followed  for  each  subsystem  in 
the  simulation  to  develop  a  complete  view  of  their 
contribution  to  the  overall  uncertainty  calculation. 


Figure  4 


4.  ECM  Captures  Uncertainty  Information 

4.1  Definition  of  Uncertainty 

According  to  the  National  Institute  of  Standards  and 
Technology  (NIST),  uncertainty  can  be  described  as: 

“The  uncertainty  (of  measurement)  parameter 
associated  with  the  result  of  a  measurement 
characterizes  the  dispersion  of  the  values  that  could 
reasonably  be  attributed  to  the  measurand.  The 
parameter  may  be,  for  example,  a  standard  deviation 
(or  a  given  multiple  of  it),  or  the  half-width  of  an 
interval  having  a  stated  level  of  confidence. 

Uncertainty  of  measurement  comprises,  in  general, 
many  components.  Some  of  these  components  may 
be  evaluated  from  the  statistical  distribution  of  the 
results  of  a  series  of  measurements  and  can  be 
characterized  by  experimental  standard  deviations. 
The  other  components,  which  also  can  be 
characterized  by  standard  deviations,  are  evaluated 
from  assumed  probability  distributions  based  on 
experience  or  other  information. 

It  is  understood  that  the  result  of  the  measurement  is 
the  best  estimate  of  the  value  of  the  measurand,  and 
that  all  components  of  uncertainty,  including  those 
arising  from  systematic  effects,  such  as  components 
associated  with  corrections  and  reference  standards, 
contribute  to  the  dispersion.  ”  (Italics  are  the 
authors’.) 

(Reference: 

http://physics.nist.gov/cuu/Uncertainty/glossary.html, 

June  27,  2001) 

4.2  PRA  Scenario  Definition 

The  PRA  scenario  consists  of  a  single  ship  steaming 
independently  when  attacked  by  a  saturation  raid  of  anti¬ 
ship  missiles.  This  is  the  situation  described  in  UML  in 
Figure  2. 

The  ability  of  a  ship  to  defend  itself  depends  upon  its 
ability  to  detect  a  threat,  track  a  threat,  and  ultimately 
destroy  the  capability  of  the  threat  to  damage  the  ship. 
The  detection  of  a  threat  consists  of  two  elements;  one  is 
the  detection  of  the  presence  of  a  potential  threat  and  next 
is  to  determine  if  the  contact  is  hostile  and  if  so  is  it  a 
threat  to  the  ship.  Once  a  contact  is  detected  the  track  of 
that  contact  must  be  maintained  until  positive 
determination  as  a  threat  is  made  and  then  the  fire  control 
system  must  be  able  to  compute  the  fire  control  solution 
and  launch  a  defensive  weapon.  This  may  be  for  a  hard 
kill,  and  electronic  attack  or  a  combination  of  the  two. 


Finally  there  must  be  an  assessment  as  to  whether  or  not 
the  threat  was  killed.  This  process  must  be  repeated  for 
each  of  the  threats  in  the  raid.  (See  Reference  7  for  more 
information  about  threat  representation  in  simulations  of 
ship  self  defense.) 

4.3  Environmental  Uncertainties 

4.3.1  Measurement 

Any  simulation  provides  many  opportunities  for 
uncertainty  but  the  focus  here  is  on  those  related  to  the 
natural  environment,  for  both  measurements  and  models. 
The  first  major  source  is  in  the  measurement  of  the 
environmental  parameters,  e.g.,  temperature,  winds  and 
currents.  Instrumentation  has  inherent  errors  as  does  the 
ability  to  record  and  report  these  measurements. 
Measurements  are  made  at  “points”  over  a  period  of  time. 
The  size  of  this  “point”  and  the  length  of  time  over  which 
the  measurement  is  made  are  sources  of  uncertainties  in 
the  value  of  a  parameter  to  be  recorded.  While  there  are 
standard  methods  for  obtaining  and  analyzing  most 
measurements,  information  regarding  the  errors  or 
uncertainties  is  usually  not  available.  Often,  only  isolated 
values  of  a  parameter  are  recorded  and  reported  for  a 
specific  place  and  time. 

4.3.2  Representation 

Measurements  of  the  ocean  and  atmosphere  environment 
have  less  obvious  limitations  as  well.  The  observational 
network  is  very  limited,  both  spatially  and  temporally. 
Measurements  are  recorded  for  a  specific  location 
(including  altitude)  and  time,  but  are  often  used  for  a 
much  larger  area  and  over  considerable  time  periods.  Are 
such  point  measurements  truly  representative?  When  a 
measurement  is  made  at  an  airport,  is  it  valid  for  a 
location  near  a  river  five  miles  away?  The  answer  to  that 
question  is  context  dependent.  For  data  represented  on  a 
grid,  seldom  are  the  measurements  taken  at  the  actual  grid 
point.  The  evaluation  of  the  importance  of  these 
difficulties  can  only  be  known  when  placed  in  the  context 
of  the  effect  on  the  system  of  interest.  This  evaluation 
must  address  the  relative  magnitude  of  the  impact  of  the 
uncertainties  of  the  system  components. 

To  counter  the  limitations  imposed  by  a  limited 
observation  network,  environmental  scientists  commonly 
use  a  judicious  combination  of  observations  and 
numerical  models,  encompassing  physical  and  dynamical 
constraints,  to  produce  natural  environment 
representations.  Again,  the  use  of  numerical  models 
imposes  uncertainties  on  the  resulting  natural 
environment  representation. 


4.3.3  Environment  Effects 

For  this  discussion  of  uncertainty,  two  environmental 
effects  are  of  primary  interest:  the  clutter  created  by  the 
surface  of  the  ocean  and  the  anomalous  propagation 
created  by  ducting  conditions.  Uncertainty  in  these 
parameters  is  introduced  during  at  least  three  events:  the 
observation  and  the  extrapolation  to  the  location  and  time 
of  interest  of  the  relevant  environmental  parameters,  and 
the  model  of  the  effect  of  these  phenomena.  This 
information  is  also  captured  in  the  Environment  Concept 
Model. 

It  is  important  in  the  discussion  and  calculation  relating  to 
uncertainty  to  determine  if  the  items  of  interest  are 
independent  or  correlated.  This  information  can  be 
documented  in  the  Environment  Concept  Model  as 
described  below.  The  method  in  which  the  uncertainty 
behavior  of  an  object  is  implemented  will  make  use  of 
this  information.  In  our  example  it  is  necessary  to  capture 


when  the  sea  state  is  correlated  with  the  anomalous  radar 
propagation  and  when  it  is  not. 

4.3.4  Documented  in  the  Environment  Concept  Model 

In  addition  to  defining  environmental  requirements  and 
providing  a  basis  for  validation  of  the  simulation,  the 
Environment  Concept  Model  provides  the  mechanism  to 
capture  and  document  the  uncertainty  and  the  relationship 
of  uncertainty  with  other  elements  of  the  system.  Within 
each  object  there  exists  attributes  and  behaviors.  The 
characteristics  of  uncertainty  may  be  unique  to  each 
object  or  entity.  Therefore  it  is  beneficial  to  allow  each 
object  to  calculate  its  own  uncertainty.  This  behavior 
stores  the  measure  of  uncertainty  as  one  of  the  objects 
attributes.  At  this  stage  of  the  work  the  uncertainty 
calculations  are  described  in  text  and  are  calculated  off 
line.  The  ability  to  instantiate  behaviors  as  computer  code 
is  a  capability  of  the  Rational  Rose  implementation  of 
UML  that  is  yet  to  be  exploited. 


Figure  5 


4.4  PRA  Uncertainty  Calculations 

The  probability  of  raid  annihilation  depends  upon  the 
ability  of  the  ship’s  systems  to  detect,  control  and  engage 
the  threat.  The  ability  of  these  systems  to  perform  is 
dependent  upon  the  nature  of  the  raid.  The  subsystems 
and  components  that  represent  the  ship’s  capabilities  are 
depicted  as  objects  in  the  Figures  above.  One  of  the 
attributes  of  each  component  captured  by  the 
Environment  Concept  Model  is  the  uncertainty  of  the 
performance  of  that  component.  The  relationship  of  each 
component  is  also  depicted  graphically.  The  ability  to 


ensure  that  each  relationship  is  considered  is  facilitated  by 
the  object  description  as  shown  in  Figure  5. 

In  the  inset  it  is  seen  that  the  SPS-48  must  consider  the 
sea  clutter  and  radar  propagation  because  it  is  a  special 
case  of  radar.  If  one  then  looks  at  the  sea  clutter  object,  in 
the  inset  in  Figure  6,  it  is  seen  that  the  sea  state  object 
provides  information  for  clutter  calculations. 

It  is  also  seen  that  sea  clutter  is  used  by  a  number  of  other 
objects  and  therefore  these  must  be  considered  when 
manipulating  the  SeaClutter  object.  This  documentation  is 


used  determine  the  degree  of  independence  of  each  of  the  As  was  stated  earlier,  the  current  process  of  evaluation 
objects  and  its  behaviors  when  evaluating  uncertainty.  depends  upon  manually  following  the  relationships  and 

calculations  contained  in  the  object  model. 


Figure  6 


4.5  Relationship  of  Environment  and  Other 
Uncertainties 

4.5.1  Scenario 

The  definition  of  a  representative  environment  for  a 
particular  scenario  remains  an  issue.  The  environmental 
conditions  used  in  the  simulation  may  be  as  big  a  factor  in 
the  system  performance  as  any  of  the  other  subsystems.  In 
the  scenario  of  interest,  if  the  potential  raid  and 
subsequent  engagement  occurs  in  an  area  of  intense  rain 
and  wind,  the  remainder  of  the  subsystems  will  most 
likely  be  of  little  consequence.  On  the  other  hand,  if  the 
raid  occurs  in  a  clear,  calm  day  with  a  moderate  sea  state 
and  no  ducting,  then  the  environment  is  of  little 
consequence.  The  uncertainty  calculations  are  related  to 
the  calculations  needed  to  evaluate  the  relative 
importance  of  the  various  subsystems.  Low  uncertainty  in 
a  subsystem  of  minor  importance  in  a  specific  scenario  is 
of  little  interest.  And  of  course,  high  uncertainty  in  a 
subsystem  of  major  importance  is  a  source  of  concern  in 
the  validity  of  a  simulation.  One  value  of  a  well-defined 
concept  model  is  to  partition  the  subsystems  into  these 
categories. 

4.5.2  System 

The  Rose  UML  description  of  the  system  allows  the 
simulation  designer  to  document  the  characteristics  of  the 


system.  This  process  allows  the  designer  to  capture 
information  from  subject  matter  experts  and  other 
documentation.  When  properly  employed  the  relationship 
among  the  subsystems  and  components  is  documented 
and  maintained  in  a  readily  available  format  in  the  object 
representations  as  seen  in  the  above  illustrations.  In  the 
context  of  this  work,  one  of  the  subsystems  is  the  natural 
environment. 

4.5.3  Example 

In  the  subset  of  the  PRA  system  provided  above  the  system 
is  tracked  through  the  ECM  from  the  high  level  view 
down  to  the  natural  environment  parameters.  Figures  3-7 
show  that  there  are  multiple  paths  from  the  ship  behavior 
down  to  one  of  the  natural  environment  parameters,  say, 
the  atmospheric  temperature.  One  such  path,  illustrated  in 
Figure  7,  can  be  demonstrated  as  follows: 

1  .  Pra  MOE  is  a  nonlinear  function  of  the 
probabilities  of  kill,  PfKy),  for  each  threat,  i,  and 
each  ship  weapon,  j . 

2.  P(Kij)  is  a  nonlinear  function  of  probabilities  of 
detection,  P(Dj),  control,  P(Q),  and  engagement 
PCEq). 

3.  P(Dj)  is  a  nonlinear  function  of  the  probability, 
d>[(sensor  k),  that  each  individual  ship  sensor, 
such  as  the  RF  (radio  frequency)  radar,  can 
detect  a  given  threat. 


4.  4VRF  radar)  is  a  nonlinear  function  of  several 
factors,  including  RF  propagation,  RP,  and  sea 
clutter,  SC. 

5 .  RP  is  a  nonlinear  function  of  number  of 
environmental  effects,  including  propagation 
loss,  PL. 


6 .  PL  is  a  nonlinear  function  of  the  natural 
environment  parameters,  including  atmospheric 
temperature,  T,  and  absolute  humidity,  H. 


Figure  7 


Only  one  path  has  been  followed  in  steps  1-6.  For 
example,  there  are  several  detection  sensors,  and 
sometimes  duplicates  of  a  given  sensor,  so  several 
different  paths  could  be  followed  in  step  4.  In  fact,  steps  2 
-  5  can  all  involve  multiple  paths.  Step  6,  however,  leads 
to  the  natural  environment  parameters,  which  are  limited 
in  number:  temperature,  pressure,  horizontal  wind  speed 
and  direction,  vertical  wind  speed,  humidity,  and  density 
as  well  as  aerosol  type  and  distribution.  For  some 
scenarios,  chemical  and  biological  agents  are  included. 
These  natural  environment  parameters  are  not 
independent  of  each  other  but  are  related  through  a  set  of 
well-known  physical  and  dynamical  relations.  As 
described  in  sections  4.4.1  and  4.4.2,  there  are 
uncertainties  in  both  the  measurements  and 
representations  of  these  parameters.  How  can  these 
uncertainties  be  related  to  uncertainty  in  the  Pra  MOE? 

The  uncertainty  in  the  atmospheric  temperature,  whether 
measured  or  modeled,  is  a  complex  and  largely  unknown 


function  of  position  and  time.  In  addition,  the  uncertainty 
in  temperature  is  related  to  the  uncertainties  in  the  other 
environmental  parameters.  Although  the  detailed 
uncertainty  functions  are  unknown,  reasonable  bounds  on 
the  uncertainties  are  available.  Then  instead  of  T  in  step  6, 
T  ±  AT  can  be  inserted  to  calculate  the  propagation  loss, 
PL  ±  APL.  The  uncertainty  in  propagation  loss  is  passed 
to  the  RF  propagation,  RP,  in  step  5  to  determine  RP  ± 
ARP.  In  step  4,  RF  propagation  uncertainty  is  passed  to 
the  probability  of  detection  for  the  given  radar  to 
determine  dL  ±  Ad^.  In  step  3,  the  uncertainty  in 
probability  of  detection  by  a  given  radar  is  passed  to  the 
total  probability  of  detection  relation  to  determine  P(D;)  ± 
AP(Dj).  And  so  on,  until  in  step  1,  Pra  MOE  ±  APra 
MOE,  is  calculated. 

The  example  described  here  tracks  only  one  part  of  the 
uncertainty  in  the  PRA  MOE  due  to  uncertainty  in 
atmospheric  temperature.  The  atmospheric  temperature 
appears  in  many  of  the  sensor  effects  algorithms  as  well 


as  in  other  military  systems  models  for  the  PRA 
Federation.  To  determine  how  the  uncertainty  in 
atmospheric  temperature  totally  affects  the  uncertainty  in 
the  PRA  MOE,  every  one  of  these  links  must  be  traced 
back  and  the  relevant  calculations  performed.  Then  it  will 
be  possible  to  state  that  uncertainty  in  atmospheric 
temperature  of  a  given  size  contributes  to  an  uncertainty 
of  a  given  size  in  the  PRA  MOE. 


Another  level  of  complexity  is  added  when  additional 
contributing  factors  at  any  stage  of  the  uncertainty 
propagation  are  considered.  When  multiple  factors  affect 
any  object,  the  proper  method  of  combining  uncertainties 
must  be  considered.  Consider,  for  example,  the  effects  of 
multiple  environmental  factors  on  propagation  loss.  We 
define  propagation  loss  (PL)  as  a  function  of  these 
environmental  factors  (E  V  ; )  as 

PL  =  f{EVuEVi,...EVn)  ■  If  we  have  expressed  the 
uncertainty  of  each  environmental  factor  as  a  standard 
uncertainty  u(EV, ),  we  can  calculate  the  standard 
uncertainty  of  the  propagation  loss,  u(PL),  as 


+  2^  Y,— — —u(EVi,EVj) 
fl^EV:  dEVj 


where  u(EVi,EVj)  is  the  covariance  associated  with  EV, 
and  EVj.  If  the  uncertainties  of  the  environmental  factors 
were  expressed  differently,  a  different  method  for 
combining  them  would  be  required. 

5.  Conclusion 

This  paper  has  described  the  use  of  the  Environment 
Concept  Model  as  a  tool  and  technique  to  identify,  trace 
and  document  the  objects  and  assumptions  associated 
with  the  evaluation  of  the  Probability  of  Raid 
Annihilation.  The  Environment  Concept  Model  provides 
the  ability  to  define  the  properties  of  systems,  models  and 
simulations  such  that  the  relationships  among  these 
entities  are  also  captured.  Furthermore,  the  ECM  can 
capture  and  document  the  specific  algorithms  used.  As 
described  here,  the  PRA  ECM  has  focused  the  natural 
environment  subsystem.  Finally,  it  has  been  demonstrated 
that  the  information  in  the  ECM  can  be  analyzed  and 
applied  to  addressing  the  uncertainties  in  the  PRA  MOE.  In 
conclusion,  the  ECM  can  provide  a  tool  for  the  tracking 
and  evaluation  of  uncertainties  in  addition  to  defining 
requirements  and  providing  a  basis  for  validation  and 
verification,  as  has  been  shown  in  earlier  paper. 
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